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Abstract 



In the search engine of Google, the PageRank algorithm plays a crucial role in ranking the 
search results. The algorithm quantifies the importance of each web page based on the link 
structure of the web. We first provide an overview of the original problem setup. Then, we 
propose several distributed randomized schemes for the computation of the PageRank, where 
^\ • the pages can locally update their values by communicating to those connected by links. The 

lO ! main objective of the paper is to show that these schemes asymptotically converge in the mean- 

^«0 ' square sense to the true PageRank values. A detailed discussion on the close relations to the 

ff^ I multi-agent consensus problems is also given. 

o 

1 Introduction 

• ^^ 

k> ' In the last decade, search engines have become widely used indispensable tools for searching the 

web. For such engines, it is essential that the search results not only consist of web pages related 
to the query terms, but also rank the pages properly so that the users quickly have access to the 
desired information. The PageRank algorithm at Google is one of the successful algorithms that 
quantify and rank the importance of each web page. This algorithm was initially proposed in [8], 
and an overview can be found in, e.g., [9ll25j. 

One of the main features of the PageRank algorithm is that it is based solely on the link structure 
inherent in the web. The underlying key idea is that links from important pages make a page more 
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important. More concretely, each page is considered to be voting the pages to which it is hnked. 
Then, in the ranking of a page, the total number of votes as well as the importance of the voters 
are reflected. This problem is mathematically formulated as finding the eigenvector corresponding 
to the largest eigenvalue of a certain stochastic matrix associated with the web structure. 

For the PageRank computation, a critical aspect is the size of the web. The web is said to 
be composed of over 8 billion pages, and its size is still growing. Currently, the computation is 
performed centrally at Google, where the data on the whole web structure is collected by crawlers 
automatically browsing the web. In practice, the class of algorithms that can be applied is limited. 
In fact, the basic power method is employed, but it is reported that this computation takes about 
a week [25]. This clearly necessitates more efficient computational methods. 

In this regard, several approaches have recently been proposed. In [22], an adaptive computa- 
tion method is developed, which classifies web pages into groups based on the speed of convergence 
to the PageRank values and allocates computational resources accordingly. Another line of re- 
search is based on distributed approaches, where the computation is performed on multiple servers 
communicating to each other. For example, Monte Carlo methods are used in [3], while the work 
in [38] utilizes the block structure of the web to apply techniques from the Markov chain literature. 
In [ISIM]; methods based on the so-called asynchronous iterations [5] in numerical analysis are 
discussed. 

In this paper, we follow the distributed approach and, in particular, develop a randomized al- 
gorithm for the PageRank computation; for recent advances on probabilistic approaches in systems 
and control, see |34j . This algorithm is fully distributed and has three main features as follows: 
First, in principle, each page can compute its own PageRank value locally by communicating with 
the pages that are connected by direct links. That is, each page exchanges its value with the pages 
that it is linked to and those linked to it. Second, the pages make the decision to initiate this 
communication at random times which are independent from page to page. This means that, in its 
implementation, there is neither a fixed order among the pages nor a centralized agent in the web 
that determines the pages to update their values. Third, the computation required for each page 
is very mild. 

The main result of the paper shows that the algorithm converges to the true PageRank values 
in the mean-square sense. This is achieved by computing the time average at each page. From 
a technical viewpoint, an important characteristic of the approach is that the stochasticity of 
the matrix in the original problem is preserved and exploited. We first propose a basic distributed 
update scheme for the pages and then extend this into two directions to enhance its performance and 
flexibility for implementation. It is further noted that in |19p20j . this approach has been generalized 
to incorporate failures in the communication as well as aggregation of the web structure. In |18j . 
a related result on finding the variations in the PageRank values when the web data may contain 
errors is given. 

We emphasize that the approach proposed here is particularly motivated by the recent research 
on distributed consensus, agreement, and flocking problems in the systems and control community; 
see, e.g., [6 l l7 1 ll0 p i4 p 2i p 23 p26l - l28l l30 p 3i p 33 p35H37] . For additional details, we refer to [Il[3l[5]. Among 



such problems, our approach to PageRank computation is especially related to the consensus, where 
multiple agents exchange their values with neighboring agents so that they obtain consensus, i.e., all 
agents reach the same value. The objective is clearly different from that of the PageRank problem, 
which is to find a specific eigenvector of a stochastic matrix via the power method. However, the 
recursion appearing in the consensus algorithm is exactly in the same form as the one for our 
distributed PageRank computation except that the class of stochastic matrices is slightly different. 
These issues will be discussed further. 

The organization of this paper is as follows: In Section [21 we present an overview of the PageR- 
ank problem. The distributed approach is introduced in Section [Sj where we propose a basic scheme 
and prove its convergence. Its relation with multi-agent consensus problems is discussed in Sec- 
tion m We then develop two extensions of the basic distributed algorithm: One in Section [5] is 
to improve the rate of convergence by allowing multiple pages to simultaneously update and the 
other in Section E] to reduce the communication load among the pages. The proposed algorithm is 
compared with an approach known as asynchronous iteration from the field of numerical analysis 
in Section [71 Numerical examples are given in Section [5] to show the effectiveness of the proposed 
schemes. We conclude the paper in Section [9l Part of the material of this paper has appeared in a 
preliminary form in |17j . 

Notation: For vectors and matrices, inequalities are used to denote entry-wise inequalities: For 
X,Y G M"^'", X < Y implies Xij < yij for i = l,...,n and j = l,...,m; in particular, we 
say that the matrix X is nonnegative if X > and positive if X > 0. A probability vector is 
a nonnegative vector v G M" such that YH=i ^« ~ 1- Unless otherwise specified, by a stochastic 
matrix, we refer to a column-stochastic matrix, i.e., a nonnegative matrix X G M"^" with the 
property that ^11=1 ^ij = 1 foi' J = li • • • ) "-• Let 1 E M" be the vector with all entries equal to 1 as 

1 := [1 • • • 1]-^. Similarly, S € M"^" is the matrix with all entries being 1. For x G M", we denote 
by |x| the vector containing the absolute values of the corresponding entries of x. The norm ||'|| for 
vectors is the Euclidean norm. The spectral radius of the matrix X G M"-^" is denoted by p{X). 
We use / for the identity matrix. 

2 The PageRank problem 

In this section, we provide a brief introductory description of the PageRank problem; this material 
can be found in, e.g., [8l[9l[25]. 

Consider a network of n web pages indexed by integers from 1 to n. This network is represented 
by the directed graph Q = (V, £). Here, V := {1, 2, . . . , n\ is the set of vertices corresponding to 
the web page indices while £" C V x V is the set of edges representing the links among the pages. 
The vertex i is connected to the vertex j by an edge, i.e., (i,i) G 8, if page i has an outgoing link 
to page J, or in other words, page j has an incoming link from page i. To avoid trivial situations, 
we assume n>2. 

The objective of the PageRank algorithm is to provide some measure of importance to each 
web page. The PageRank value, or simply the value, of page i G V is a real number in [0, 1]; we 



denote this by x*. The values are ordered such that x* > x* imphes that page i is more important 
than page j. 

The basic idea in ranking the pages in terms of the values is that a page having links from 
important pages is also important. This is realized by determining the value of one page as a sum 
of the contributions from all pages that have links to it. In particular, the value x* of page i is 
defined as 



xj 

Hi 



where Ci := {j : (j, i) € £}, i.e., this is the index set of pages linked to page i, and Uj is the number 
of outgoing links of page j. It is customary to normalize the total of all values so that XlILi ^i ~ ^■ 
Let the values be in the vector form as x* € [0, 1]". Then, from what we described so far, the 
PageRank problem can be restated as 



x* = Ax*, x*e[0,l]", J^x* = l, (1) 

1=1 

where the matrix A = (aij) € M"^*^, called the link matrix, is given by 

a.r-=i^ if^'^^- (2) 

I otherwise. 

Note that the value vector x* is a nonnegative unit eigenvector corresponding to the eigenvalue 1 of 
the nonnegative matrix A. In general, for this eigenvector to exist and to be unique, it is sufficient 
that the web as a graph is strongly connected |15)^l . However, the web is known not to be strongly 
connected. Thus, the problem is slightly modified as followqj. 

First, note that in the real web, the so-called dangling nodes, which are pages having no links to 
others, are abundant. Such pages can be found, e.g., in the form of PDF document files having no 
outgoing links. These pages introduce zero columns into the link matrix. To simplify the discussion, 
we redefine the graph and thus the matrix A by bringing in artificial links for all dangling nodes 
(e.g., links back to the pages having links to a dangling node). As a result, the link matrix A 
becomes a stochastic matrix, that is, X]"=i ciij = 1 for each j. This implies that there exists at 
least one eigenvalue equal to 1. 

To emphasize the changes in the links that we have just made, we state the following as an 
assumption. 

Assumption 2.1 The link matrix A given in ([2]) is a stochastic matrix. 



^ A directed graph is said to be strongly connected if for any two vertices i, j £ V, there is a sequence of edges which 
connects i to j. In terms of the hnk matrix A, strong connectivity of the graph is equivalent to A being irreducible. 

^In fact, in the consensus literature, it is known that the eigenvalue 1 of a row-stochastic matrix is simple if and 
only if the underlying graph has at least one globally reachable node; this means that there is a node from which 
each node in the graph can be reached via a sequence of edges (see, e.g., [261I281I5T] '). For our purpose, it is indeed 
possible to provide the column-stochastic counterpart of global reachable nodes, but the real web does not possess 
this property either. 



Next, to guarantee the uniqueness of the eigenvalue 1, a modified version of the values has been 
introduced in [5] as follows: Let m he a parameter such that m G (0, 1), and let the modified link 
matrix M G R"^" be defined by 

777 

M ■.= (l-m)A + —S. (3) 

n 

In the original algorithm in [8], a typical value for m is chosen as m = 0.15; we use this value 
throughout this papeio. Notice that M is a positive stochastic matrix. By the Perron theorem |15] . 
this matrix is primitivqj; in particular, the eigenvalue 1 is of multiplicity 1 and is the unique 
eigenvalue of maximum modulus (i.e., with the maximum absolute value). Furthermore, the cor- 
responding eigenvector is positive. Hence, we redefine the value vector x* by using M in place of 
^ in ([T]) as follows. 

Definition 2.2 The PageRank value vector x* is given by 

n 

x* = Mx\ x*G[0,l]", ^x* = l. (4) 

As mentioned in the Introduction, due to the large dimension of the link matrix M, the com- 
putation of the eigenvector corresponding to the eigenvalue 1 is difficult. The solution that has 
been employed in practice is based on the power method. That is, the value vector x* is computed 
through the recursion 

77? 

x{k + l) = Mx{k) = {l-m)Ax{k) + —l, (5) 

n 

where x{k) G M" and the initial vector x(0) G M"' is a probability vector. The second equality above 

follows from the fact Sx{k) = 1, /c G Z^_. For implementation, the form on the far right-hand side 

is important, exhibiting that multiplication is required using only the sparse matrix A, and not the 

dense matrix M. 

Based on this method, we can asymptotically find the value vector as shown below; see, e.g., |15j . 

Lemma 2.3 In the update scheme ([5]), for any initial state x(0) that is a probability vector, it 
holds that x{k) — > x* as A; ^- co. 

We now comment on the convergence rate of this scheme. Denote by Ai(M) and A2(M) the 
largest and the second largest eigenvalues of M in magnitude. Then, for the power method applied 
to M, the asymptotic rate of convergence is exponential and depends on the ratio |A2(M)/Ai(M)|. 
Since M is a positive stochastic matrix, we have Ai(M) = 1 and |A2(M)| < 1. Furthermore, it is 
shown in [25] that the structure of the link matrix M leads us to the bound 



|A2(M)|<l-m. (6) 

We next provide a simple example for illustration. 



^In [8], no specific reason is given for tliis ciioice of m. As siiown later in ((6]), liowever, large m has the effect of 
faster convergence in the computation while it can also average out the PageRank values. 

*A nonnegative matrix X G R"^" is said to be primitive if it is irreducible and has only one eigenvalue of maximum 
modulus. 




Figure 1: A web with four pages 

Example 2.4 Consider the web with four pages shown in Fig. [TJ As a graph, this web is strongly 
connected, and there are no danghng nodes. The hnk matrix A and the modified hnk matrix M 
can easily be constructed by ([2]) and ([3]), respectively, as 



A 







1 

2 

1 

2 



? 



M 



0.0375 
0.8875 
0.0375 
0.0375 



0.0375 
0.0375 
0.4625 
0.4625 



0.0375 
0.4625 
0.0375 
0.4625 



0.3208" 
0.3208 
0.3208 
0.0375 



where we used the value m = 0.15 from [8]. Then, the value vector x* can be computed as 
X* = [0.119 0.331 0.260 0.289] . Notice that page 2 has the largest value since it is linked from 
three pages while page 1, which has only one link to it, has the smallest value. On the other hand, 
pages 3 and 4 have the same number of incoming links, but page 4 has a larger value. This is 
because page 4 has more outgoing links, and thus it receives more contribution from page 3 than 
what it gives back. V 



3 A distributed randomized approach 

In this section, we propose a distributed approach to compute the value vector x*. 

Consider the web from the previous section. The basic protocol of the scheme is as follows: At 
time k, page i initiates its PageRank value update (i) by sending the value of page i to the pages 
that are linked and (ii) by requesting the values from the pages that are linked to page i. All pages 
involved in this process renew their values based on the latest available information. 

To implement the scheme in a distributed manner, we assume that the pages taking the update 
action are determined in a random manner. This is specified by the random process 9{k) € V, 
k £ Z_|_. If at time k, 9{k) = i, then page i initiates an update action by communicating and 
exchanging the values with the pages connected by incoming and outgoing links. Specifically, 6{k) 
is assumed to be i.i.d., and its probability distribution is given by 



Fi-oh{e{k) = i} = -, ykeZ+. 



n 



(7) 



This means that each page takes the update action with equal probability. In principle, this scheme 
may be implemented without requiring a centralized decision maker or any fixed order among the 
pages. 



In particular, consider the distributed update scheme in the following form: 



+ 1) 



(1 - m)AQ(^k)x{k) 



m 



n 



(8) 



where x{k) € M" is the state whose initial state x[{)) is a probability vector, 9[k) € {1, . . . ,n} is 
the mode of the system, and Ai, i = 1, . . . ,n, are called the distributed link matrices and are to be 
determined; m € (0, 1) is a parameter replacing m in the centralized scheme ([5]). 

The objective here is to design this distributed update scheme by finding the appropriate link 
matrices Ai and the parameter rh so that the PageRank values are computed through the time 
average of the state x. Let y{k) be the average of the sample path x(0), . . . , x{k) as 



y{k) :-- 



1 



k 

k + 1 ^ 



x{i). 



(9) 



We say that, for the distributed update scheme, the PageRank value x* is obtained through the 
time average y if, for each initial state x(0) that is a probability vector, y{k) converges to x* in the 
mean-square sense as follows: 

*l|2 



E 



yik) 



0, k 



oo. 



(10) 



This type of convergence is called ergodicity for stochastic processes [29] . 

In what follows, we develop the distributed update scheme of dS]). The main result is presented 
as Theorem 13.41 showing the convergence of the scheme. The design consists of two steps, one for 
the link matrices Ai and then the parameter rh. In later sections, this approach will be extended 
to improve the convergence rate and the necessary communication load. 

3.1 Distributed link matrices and their average 

The first step in the development is to introduce the distributed link matrices. For each i, the 
matrix Ai € M"^" is obtained as follows: (i) The ith row and column coincide with those of A; (ii) 
the remaining diagonal entries are equal to 1 — an, i = 1, . . . ,n, £ y^ i; and (iii) all the remaining 
entries are zero. More formally, we have 



{Ai)je ■■-- 



(11) 



Qji if j = i or i = i, 

l-an [ij = i^i, i = l,...,n. 

otherwise, 

It follows that these matrices are stochastic because the original link matrix A possesses this 
property. As we shall see later, this property indeed is critical for the convergence of the scheme. 

Example 3.1 We continue with the 4-page web in Example 12.41 The link matrices Ai are given 
by 



^1 












l-l 

3 




"0 











1 


1 










1 





1 


1 








1 





, A2 = 





1 

2 


! 

2 


3 













2 







1 

2 





2 
3 



A. 



i 







0" 




"1 








1 

! 


1 










1 





2 




1 


, A4 = 





2 




1 


! 

3 


1 

2 


3- 







1 

2 


1 

2 






V 



To clarify the properties of the hnk matrices Ai just introduced, we consider the simpler update 
scheme 

xik + l) = Ae^k)x{k), (12) 

where x{k) is the state with x{0) being a probability vector, and the mode 9{k) is specified in 
([7]). In particular, we focus on its average dynamics. The mean x{k) := E[x{k)] of the state x{k) 
follows the recursion x{k + 1) = Ax{k), where A := E[A0ri^\] is the average matrix and E[-] is the 
expectation with respect to the random process 9{k). Hence, we now inspect this matrix A. Due 
to the probability distribution of 9{k) in ([7]), we have 

A = -^A,. (13) 

It is obvious that A is a stochastic matrix since all Ai are stochastic. 

The following lemma shows some properties of this matrix A that will be useful later. 

Lemma 3.2 For the average matrix A given in (|13|) . we have the following: 

(i) A = ^A + ^^/. 

(ii) There exists a vector zq G M" which is an eigenvector corresponding to the eigenvalue 1 for 
both matrices A and A. 

Proof: (i) By definition of A, we have 

'^hi + ELi, i^i(l-aii)] 



_ 1 " 
{Ay = - ^(^, 

i=\ 



By definition of A^ we have Oj-j = and 'Y^=\ i^j ^«i ~ 1- Thus, the expression for A follows. 

(ii) From (i), we have A — I = 2/n(A — I). This implies that every eigenvector zq of the link 
matrix A associated with the eigenvalue 1 is also an eigenvector of the average matrix A for the 
same eigenvalue. D 

The lemma above provides some justification for the proposed distributed approach. That is, 
even though the matrices A and A have different structures, they share an eigenvector for the 
eigenvalue 1, which corresponds to the PageRank vector. 

3.2 Mean-square convergence of the distributed update scheme 

As in the case with the original link matrix A, for the average matrix A, the eigenvector corre- 
sponding to the eigenvalue 1 may not be unique. We follow an argument similar to that in Section [2] 
and introduce the modified versions of the distributed link matrices. 

Since the link matrices Ai are stochastic, we can rewrite the distributed update scheme in ^ 
as 

x{k + l) = Me^k)xik), (14) 

8 



where the modified distributed link matrices are given by 

771 

Mi:= (l-m)Ai-\ S, i = l,...,n. (15) 

n 

This expression is derived by the relation Sx{k) = 1 because x{k) in dSD is a probabihty vector for 

each k. Note that Mj are positive stochastic matrices. 

Similarly to the argument on the link matrices Ai above, the problem at this second step is as 
follows: We shall find the modified link matrices Mj by choosing the parameter m such that their 
average and the link matrix M from ([3]) share an eigenvector for the eigenvalue 1. Since such an 
eigenvector is unique for M, it is necessarily equal to the value vector x* (see (j3|)). 

Let x{k) := E[x{k)] be the mean of the state x{k) of the system ()14p . Its dynamics is expressed 
as 

x{k + l) = Mx{k), (16) 

where x(0) = rc(0) and the average matrix M is given by M := EIMqi/.^]. 

A simple way of defining Mi would be to let m = m as in the case with M; however, it can 
be shown that there is no clear relation between the original matrix M and the average matrix M 
such as that between A and A as we have seen in Lemma [3.2[ Instead, we take the parameter m as 

2m 

rh = -. (17 

n — m[n — 2) 

For the value m = 0.15 used in this paper, we have m = 0.3/(0.85n + 0.3). For this choice of m, 
the next result holds. 

Lemma 3.3 For the parameter m given in (J17p . we have the following: 

(i) m £ (0, 1) and rh < m. 

(ii) M = ^M+ (1-^)/. 

(iii) For the average matrix M, the eigenvalue 1 is simple and is the unique eigenvalue of maximum 
modulus. The value vector x* is the corresponding eigenvector. 

Proof: (i) By the assumptions m € (0, 1) and n > 2, m in p7|) is positive. Also, we have 
1 — rh = n{\ — m)/[n — m{n — 2)]. Hence, 1 — m > 1 — m, that is, m < m <1. 
(ii) This can be shown by direct calculation as follows: 

M = (1 - rh)A H 5 (by the definition of M, ([15]), and then (fT3|) ) 

n 



(1 — rh) 



2 ^ n-2^ 

-A + / 

n n 



m ^_^_ 

+ —S (by Lemma [32](i)) 
n 



in I in \ ^^ 

= —M +11 j / (by m in ([HD and the definition of M in ([3])). 

(iii) From (ii), we have M — I = rh/m[M — I). Hence, M and M share an eigenvector for the 
eigenvalue 1. However, both M and M are positive stochastic matrices. Therefore, by the Perron 

9 



theorem [15] . for these matrices, the eigenvalue 1 is of multiphcity 1 and is the unique one having 
the maximum magnitude. Moreover, by (JU, the corresponding eigenvector coincides with x* . D 

From (iii) in the lemma above, it follows that the value vector x* can be obtained by the 
power method, i.e., by the average system in ()16p as x{k) -^ x* , k — ?> oo. Hence, in an average 
sense, the distributed update scheme asymptotically provides the correct values. It is interesting 
to observe that this can be achieved though the original link matrix A does not explicitly appear 
in the scheme. In fact, an eigenvector of the matrix M is computed through randomly switching 
among the distributed link matrices Mj. 

However, this property turns out not to be sufficient to guarantee convergence of x{k) to the 
true value x* . From Lemma lA. 41 (ii) in the Appendix, we can show that for any sequence {6{k)}, 
there exists a sequence of probability vectors {v{k)} such that, for any x{0), it holds that x{k) — 
v{k)l'^x{0) = x[k) — v{k) — 7> as A; — )■ oo. The vector v and hence the state x in general do not 
converge. This can be seen in Example 13.11 when, e.g., page 1 initiates an update {6{k) = 1); the 
update for page 4 is given by X4^{k + 1) = 2(1 — m)/3x4(fe) + m/4, showing that X4 cannot stay 
at its equilibrium value. Therefore, in the distributed approach, we resort to computing the time 
average y{k) of the states. 

The following theorem is the main result of this section. It shows that the time average indeed 
converges to the value vector in the mean-square sense. 

Theorem 3.4 In the distributed update scheme in ()14p . the PageRank value x* is obtained through 
the time average y in ([U]) as i?[||y(A;) — j;*|| 1 — > 0, fc ^- co. 

The theorem highlights an ergodic property in the proposed update scheme. It can be shown 
by general Markov process results in, e.g., [11]. For completeness, however, a proof more specific 
to the current setup is provided in Appendix IA.21 it employs tools for stochastic matrices and 
moreover is useful for an extension given in Section [6l Regarding the convergence of this algorithm, 
we see from (|53|) in the proof that it is of order \/k and moreover depends on the size of n linearly 
through the parameter rh in (|17p . 

Several remarks are in order. In practice, each page needs to communicate with the pages that 
are directly connected by incoming or outgoing links. We emphasize that the recursion to be used 
is ([8]) and not the equivalent expression of ()14p : in the latter case, the link matrices Mi are positive, 
which can imply that the values of all pages are required for an update of a page. Nevertheless, as 
can be seen in ([8]), the link matrices Ai are sparse. Thus, at time k, communication is required only 
among the pages corresponding to the nonzero entries in AQn.\ . Each page then performs weighted 
addition of its own value, the values just received, and the extra term rh/n. Consequently, the 
amount of computation required at each page is limited at any time. 

Implementation issues such as how web pages can exactly make local computations are outside 
the scope of this paper. However, it is clear that certain regulations may be necessary so that 
page owners cooperate with the search engine and the PageRank values computed by them can be 
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trusteqj. Another issue concerning reliability of the ranking is that of link spam, i.e., links added to 
enhance the PageRank of certain pages on purpose; a method to detect such spamming is studied 

in, e.g., mm- 

4 Relations to consensus problems 

In this section, we discuss the relation between the two problems of PageRank and consensus. First, 
we describe a stochastic version of the consensus problem. Such problems have been studied in, 
e.g., [ZlElEalEB]; see also [35]. 

Consider a set V = {1,2, ... ,n} of agents having scalar values. The network of agents is 
represented by the directed graph Q = {V,£). The vertex i is connected to the vertex j by an 
edge {i,j) € f if agent i can communicate its value to agent j. Assume that the graph is strongly 
connecteqj. 

The objective is that all agents reach a common value by communicating to each other, where 
the pattern in the communication is randomly determined at each time. Let Xi(k) be the value of 
agent i held at time k, and let x{k) := [xi{k) ■ ■ ■ Xn{k)] G M". The values are updated via the 
recursion 

x{k + l) = Ag(^k)x{k), (18) 

where 0{k) € {1, . . . , d} is the mode specifying the conmiunication pattern among the agents and d is 
the number of such patterns. The communication patterns are given as follows: Each i £ {1, . . . ,d} 
corresponds to the subset £i C £ oi the edge set. Then, the matrix Ai has {Ai)j£ > if and only 
if {i,j) € £i. We assume that (i) {j,j) G £i for all j, (ii) IJj^i £i = £, and (iii) the matrix Ai is a 
row-stochastic matrix. The communication pattern is random, and in particular, 0{k) is an i.i.d. 
random process. Its probability distribution is given by Prob{0(fc) = i} = 1/d for k € Z_|_. 
We say that consensus is obtained if for any initial vector x{0) G M", it holds that 

\xi{k) - Xj{k)\ ^ 0, k^oo (19) 

with probability one for all i,j € V. 

A well-known approach is to update the value of each agent by taking the average of the values 
received at that time. In this case, the matrix Ai is constructed as 

[O otherwise, 

where rijj is the number of agents i with {i,j) € £i, i.e., those that transmit their values to agent 
3- 



^In the consensus literature, problems involving cheating have been studied. An example is the Byzantine agree- 
ment problem, where among the agents there are malicious ones who send confusing information so that other agents 
cannot achieve consensus (see, e.g., [35]'). 

® As discussed in Section [21 this assumption can be replaced with a weaker one that a globally reachable node 
exists. 
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Example 4.1 Consider the graph in Example 12.41 with four agents. We introduce four communica- 
tion patterns arising from the protocol in the distributed PageRank algorithm: The edge subset 8i 



contains all (i, j) and (j, i) in the original edge set 8 including 
for i,j = 1,2,3,4. The matrices Ai can be written as 



i, i) that corresponds to a self-loop 



Ai 



i 

2 








i 
2 


1 

2 


1 

2 














1 














1 



Ao 



1 











1 


1 


1 


1 


4 


4 


4 


4 





1 

2 


1 

2 








1 
2 





1 

2 



A. 



1 














1 

2 


1 

2 





n 


1 


1 


1 





3 



? 

2 


! 

2 



A, 



1 

2 








1 
2 





1 

2 





1 

2 








1 

2 


1 
2 





1 

3 


1 

3 


1 
3 



V 



We now present the convergence result for consensus. 

Lemma 4.2 Assume that the graph is strongly connected. Then under the scheme of (jlSp . where 
the communication pattern is chosen randomly, consensus is obtained in the sense of (|19p . 



Outline of the proof: Let A := E[Aqij^\\ be the average matrix. This matrix is stochastic 
and irreducible. This is because the original graph is strongly connected, and hence under the 
probability distribution of 9{k), we have {A)ji > for each {i,j) € £. Furthermore, by definition, 
the diagonal entries are positive, and thus ^4 is a primitive matrix, implying that it has the unique 
eigenvalue 1 of maximum modulus [TS]. Thus, by [33] > it follows that consensus is obtained. D 

In comparison with the distributed PageRank problem, the consensus problem has the features 
below: 

(i) The graph is assumed to be strongly connected. 

(ii) The goal is that all values Xi{k) become equal, and moreover there is no restriction on its 
size. 

(iii) Convergence with probability one can be attained for the values Xi(k) directly; there is no 
need to consider their time average (as in yi{k) in ([9])). 

(iv) The matrices Ai are row stochastic and the diagonal entries are all positive. In contrast, 
in our distributed PageRank computation scheme, the link matrices are column stochastic. 
However, the coefficient of ergodicity, which is the tool employed for proving Theorem 13. 4t is 
useful also for this problem; see, e.g., |33j . 

It is clear that many similarities exist between the algorithms for consensus and PageRank. We 
emphasize that the distributed PageRank approach in this paper has been particularly motivated 
by the recent advances in the consensus literature. We highlight two points that provide helpful 
insights into the PageRank problem as follows: 

(1) At the conceptual level, it is natural to view the web as a network of agents that can make 
its own computation as well as communication with their neighboring agents. 

(2) At the technical level, it is important to impose stochasticity on the distributed link matrices. 
For the distributed PageRank computation, very few works exploit this viewpoint. 
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5 Extensions to simultaneous updates 

So far, we have discussed the update scheme where only one page initiates an update at each time 
instant. In the web with bilhons of pages, however, this approach may not be practical. In this 
section, we extend the distributed algorithm by allowing multiple pages to simultaneously initiate 
updates. 

Consider the web with n pages from Sections [2] and [3l As before, at each time k, the page i 
initiates its PageRank value update (i) by sending its value to the pages that it is linked to and (ii) 
by requesting the pages that link to it for their values. All pages involved here update their values 
based on the new information. The difference from the simpler scheme before is that there may be 
pages that are requested for their values by multiple pages at the same time. The current scheme 
handles such situations. 

These updates can take place in a fully distributed and randomized manner. The decision to 
make an update is a random variable. In particular, this is determined under a given probability 
a € (0, 1] at each time k, and hence, the decision can be made locally at each page. Note that the 
probability q is a global parameter in that all pages share the same a. 

Formally, the proposed distributed update scheme is described as follows. Let r]i{k) € {0, 1}, 
I = 1, . . . , n. A; G Z+, be Bernoulli processes given by 

J 1 if page i initiates an update at time k, 
1 otherwise, 

where their probability distributions are specified as 

a:=Prob{r?i(A;) = 1}. (20) 

The process r]i{k) is generated at the corresponding page i. 

Similarly to the argument in Section [31 we start with the update law as in ([5]) : 

TTl 

x{k + 1) = (1 - m)A^^(^k),...,r,r.{k)x{k) -\ 1, (21) 

where x{k) € M"", the initial state x(0) is a probability vector, m € (0, 1) is the parameter used 
instead of m in the centralized case, and A.^i-^{k),...,rin(k) ^-re the distributed link matrices. 

The problem of distributed PageRank computation is formulated as follows: Find the dis- 
tributed link matrices Aj,^^,,,.p,j and the parameter m such that, for the corresponding distributed 
update scheme ([2T]) . the PageRank value x* is obtained through the time average. This problem 
is a generalization of that in Section El where only one page initiates an update at a time. The 
current approach is called the distributed scheme with simultaneous updates. Its analysis is more 
involved as we shall see. 
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5.1 Distributed link matrices and their average 

We introduce the distributed link matrices. Let the matrices Ap^^...^p^ be given by 

'ij if Pi = 1 or pj = 1, 



A 



Pl,--;Pn, 



«J 



1 - Eh: pft=i '^hj if Pi = and i=j, (22) 

if Pi = Pj = and i ^ j 



for pr G {0, 1}, r S {1, . . . ,n}, and i,j € {1, • • • ,n}. Clearly, there are 2" matrices. They have 
the property that (i) if pi = 1, then the ith column and the ith row are the same as those in the 
original link matrix A, (ii) if pi = 0, then the ith diagonal entry is chosen so that the entries of the 
ith column add up to 1, and (iii) all other entries are 0. Hence, these are constructed as stochastic 
matrices. Notice that the link matrix Ap^^...^p^ coincides with Ai in ()lip when pi = 1 and pj = 
for all j ^ i. 

We next analyze the average dynamics of the distributed update scheme in (|2ip . For simplicity, 
as in Section [31 we use the same notation A for the average link matrix given by 

A:= E[A^^^k),...,r,^{k)\, (23) 

where E[-] is the expectation with respect to 'r]i{k), i € V. This matrix A is stochastic. 

The following result shows that the average link matrix A has a clear relation to the original 
link matrix A. In particular, it implies that the two matrices share the eigenvector for eigenvalue 
1. 

Proposition 5.1 For the average link matrix A given in (j23p . we have the following: 

(i) A= [l-(l-Q)2]A + (l-a)2/. 

(ii) There exists a vector zq E M" which is an eigenvector corresponding to the eigenvalue 1 for 
both A and A. 

The proof of this proposition is preceded by a preliminary result. Observe that A can be written 



as 



A = Y,a'{l-ar-'Ae, (24) 



where the matrices Ag, i = 0,1, . . . ,n, are given by 

Ae.= Yl \u-,Pn- (25) 

Pre{0,l}, r=l,...,n: 

The matrix Ai is the sum of matrices for the cases where i pages simultaneously initiate updates. 
These matrices can be explicitly written in terms of the original link matrix A. Here, we use 
the binomial coefficient given by rCk '■= r\/[{r — k)] k\]. Note that j-Cq = 1 for any r G Z-|_. 
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Lemma 5.2 The matrices A^, i = 0,1, . . . ,n, in ()25p can be expressed as follows: 

'A \ii = n, 

h=\nA \ii = n-\, (26) 

_ [nCt - n~2Ci)A + n-2CiI if i = 0, . . . ,n - 2. 

Proof: We consider four cases as follows. 

(1) £ = n: This is the case when all pages initiate updates, and thus by definition An = ^i,...,i = 
A. 

(2) i = n — 1: When all but one page initiate updates, it is obvious from the definition that 
^0,1,. ..,1 = ^1,0,1,...,! = • • • = ^i,...,i,o = A. Since there are n such cases, their sum is A^-i = nA. 

(3) ^ = 0: In the case when none of the pages initiates an update, by definition, the matrix 
^o,...,o reduces to the identity matrix as Aq = ^o,...,o = I- Noting that ,„Co = n-2C'o = 1, we have 

(ESD. 

(4) ^ = 1, . . . , n — 2: To prove the expression of A^ for these cases, we must show for each entry 
that 

( A\ _ I ("^^ ~ n-2Ci)aij if i ^ j, 

i,je{l,...,n}. (27) 

In the following, the proof is divided into two steps for the cases of i ^ j and i = j- 
(i) i 7^ j: By the definition of ^pi,...,p„ in ([22]) . its {i,j) entry reduces to 

/^ . ^ (aij if K = 1 or pj = 1, 

" ^^ 1 otherwise. 



Thus, from (|25 



Pr6{0,l}, r=l,...,n: 



E 



Clij 



PrG{0,l}, r=l,...,n: 
E"=iPr=£ and 
(pi=l or Pj=l) 

= {2n-iCe-i — n-2Ce-2)aij, (28) 

where the last equality is established by counting the number of cases where pj = 1 or pj = 1 holds 
among all possible combinations of pi, ... ,pn € {0, 1} such that £ of them are equal to 1. Using 
the formula for binomial coefficients 

rCk = r-lCk + r-lCk-l, r,k G Z+, (29) 

we can show that 2„_iC£_i — n-2Ci-2 = nCe, — n-2Ce,. Hence, from ([28]) . we arrive at (p7|) for i ^ j. 
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(ii) i = j: Since an = in the link matrix j4 in ([2]) and by ()22p . the {i,i) entry of j4p^^...^p^ is 



Hence, 

Pre{0.1}, r=l,...,n: prG{0,l}, r=l....,n.: ^ h: ph=l 

n 

= n-lQ — ^ ^ O/ii, 

h=l pre{0,l}, r=l,...,n: 

where the first term is obtained by counting the number of possible combinations of pi, . . . ,pn such 
that their sum equals i and pi = 0; the second term is a consequence of switching the order of two 
summations. By a combinatorial argument again, we have [AA .. = n-iCe. — n~2Ci-i X^/J=i h^i ^hi- 
The original link matrix yl in ([2]) is stochastic with diagonal entries being 0. This fact together 
with (pU|) yields [Ai) .. = n-iCi — n-2Ci^i = n-2C£. Therefore, ([TTjl is attained for the case i = j. 

D 

Proof of Proposition [5!7l " (i) By ()24p and Lemma 15.21 the matrix A can be computed directly 
as 



n-2 

= Y^ a\l - ar-'[{nCt - n-2Ce)A + „_2Q/] + a^-\l - a)?iA + aM 

i=Q 
n n—2 

= Y.a\l - aT-'nCiA + Y,o^\l - a)"-V2Q(/ - A). (30) 

In the first term above, we have by the binomial identity Y^^=o ck^(l ~ Oi)"'~^nCi = [a + (1 — a)]" = 1. 
Similarly, for the second term, Yll=o '^^(^ ~ (^)"'~^n-2Ce = (1 ~ ct)^- Substituting these relations 
into ()30p . we have A = A + {1 — a)'^{I — A), which is the desired expression of A. 

(ii) The equality in (i) implies that any eigenvector zq of the link matrix A associated with the 
eigenvalue 1 is also an eigenvector of the average matrix A for this eigenvalue. D 

5.2 Mean-square convergence of the distributed update scheme 

To guarantee that the distributed scheme yields the PageRank value, we now examine the modified 
versions of the distributed link matrices. We first express the distributed update scheme of ()2ip in 
its equivalent form as 

x{k + 1) = M^^{k),...,r,n{k)x{k), (31) 
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where the modified distribution link matrices are given by 

TTl 

Mp,,...,p„ := (1 - m)Ap,,...,p„ + -5, pu ... ,Pn e {0,1}- (32) 

This form can be obtained by using the facts that the hnk matrices Ap-^^^^^^p^ are stochastic and 
that Sx{k) = 1. Clearly, these matrices Mp^^...^p„ are positive and stochastic. 

The objective here is to find the modified link matrices -Mp^^...^^^, by selecting the parameter 
m, so that their average and the link matrix M from ([3|) share an eigenvector corresponding to the 
eigenvalue 1. Since such an eigenvector is unique for M, it is necessarily equal to the value vector 

X*. 

As in the earlier case in Section [3l we take the parameter rh to be different from the original 
m. In particular, let 

. m[l-{l-af] 

^ = 1 n yT- (33 

1 — m[l — a)^ 

For the value m = 0.15 used in this paper, we have rh = 0.15[1 — (1 — a)^]/[l — 0.15(1 — a)^]. 
Then, let the average link matrix be M := E[Af^j/fc> ,^^(fc')]. Here, the distributed link matrices 
are positive stochastic matrices, which implies that the average matrix M enjoys the same property. 
The next lemma is the key to establish the desired relation between the distributed link matrices 
and their average. It is stated without proof; it follows similarly to that for Lemma 13.31 



Lemma 5.3 The scalar m in (j33p and the link matrices Mp^^...^p^ in (j32p have the following prop- 
erties: 

(i) m € (0, 1) and rh < m. 

(ii) M = ^M+ (1-^)1. 

(iii) For the average matrix M, the eigenvalue 1 is simple and is the unique eigenvalue of maximum 
modulus. The value vector x* is the corresponding eigenvector. 

We can show by (iii) in the lemma that, in an average sense, the distributed update scheme 

/j 

asymptotically obtains the correct values. More precisely, we have E[x{k)] = M x{0) — )■ x* as 

fe — > oo. Further, as discussed in Section [21 the asymptotic rate of convergence is dominated by the 

second largest eigenvalue \2{M) in magnitude. By ([6]) and (ii) in the lemma, this eigenvalue can 

be bounded as 

w,,^.,^^, "rh 1 — m 

A2 M = - A2 M + 1 - - < ^. 

m ml — m[l — ay 

It is clear that this is a monotonically decreasing function of a and m. That is, higher probability 
in updates and/or larger m results in faster convergence in average. 
We are now ready to state the main result of this section. 
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Theorem 5.4 Consider the distributed scheme with simultaneous updates in ()3ip . For any update 
probability a G (0,1], the PageRank value x* is obtained through the time average y in Q as 

The proof follows along similar lines as that in Theorem 13.41 More specifically, we can prove 
either by the general Markov chain results of, e.g., [TT] or by Appendix IA.2I where we replace the 
expression of M there with the one in Lemma 15.31 Hence, the convergence is of order l/Zc, as in 
the algorithm of Section O it also depends on the update probability a but is independent of n. 

We remark that this scheme is fully decentralized when a < 1. It is parameterized by a, 
which determines the frequency in the updates, communication load among the pages, and the 
rate of convergence in the mean as we have seen above. In practice, the recursion in (|3ip must be 
implemented in the equivalent form ()2ip . It is clear that communication is required only over the 
links corresponding to the nonzero entries in the link matrices there. Each page then computes 
weighted additions of its own value, the values received from others, and the constant min. On 
the other hand, when a = 1, the scheme reduces to the original centralized one in Section [21 In 
this case, the distributed link matrix is Mi^,,.^i and coincides with the original M because m = m 
and ^i,...,i = A from Lemma |5. 21 

6 Update termination in PageRank computation 

In this section, we further develop the distributed algorithm for calculating the PageRank. We 
relax the objective and aim at obtaining approximate values of the PageRank. The key feature 
here is to allow the pages to terminate their updates at the point when the values have converged 
to a certain level. The benefit is that such values need to be transmitted only once to the linked 
pages; hence, the computation and communication load can be reduced. 

In a centralized setting, the idea of update termination for the PageRank computation has been 
introduced by [22]. We extend this idea to the distributed update scheme of Section [S] First, we 
consider a simple case to attain a convergence result. Then, we provide the details of the proposed 
algorithm. 

6.1 Convergence properties for the distributed scheme 

Consider the distributed update scheme with simultaneous updates in (|2ip for computing the 
values x{k) together with their time average y{k). Within this subsection, we fix the sample paths 
{^j(^)}fc=^ , j = 1, . . . ,n, up to time /cq — 1 of the processes specifying the updates in the pages. 
Suppose that some of the time averages 2/i(A;o) have, in an approximate sense, converged. This 
is measured by finding those that have varied only within sufficiently small ranges for a certain 
number of time steps. We introduce two parameters: Let 6 G (0, 1) be the relative error level, 
and let Ng be the number of steps. Using the history of its own time average yi, each page i then 
determines at time ko whether the following condition holds: 

\yi{ko)-y^iko-i)\<6y^{ko), i = l,2,...,Ns. (34) 
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If so, then (i) the page i will terminate its update and fix its estimate at yj(fco), and then (ii) this 
value yiiko) is transmitted to the pages connected to i by direct links. After this point, these values 
will be used at the pages performing further updates. 

The question of interest is whether the pages that continue with their updates after time ko will 
reach a good estimate of their true values. In what follows, we show that the answer is positive and 
the approximation level achievable in the estimate will be as good as that for the pages that have 
terminated their updates. Note that the analysis is based on the given sample paths {'nj{k)}f.°SQ , 
and hence the state x and its average y up to time fco are fixed; we study their stochastic behaviors 
after this time. 

Let C{k()) be the set of page indices that have reached good estimates at time ko as 



C{ko) ■.= {ieV: Mko) - viiko -i)\< 6y^{ko), i 



1,2, 



,Ns}. 



x{k) 



k > h 



0, 



The cardinality of this set is denoted by ni{ko). We assume ni(A;o) > 1- Also, let M{kQ) := V\C(/co). 

Based on these sets, we introduce a coordinate transformation for the state x{k) and partition 

it as 

xc{k) 

XAf{k)_ 

where xc{k) G M^-iC^o) contains the values of the pages in C(A:o) and xj\f{k) € M"~"i('^o) contains 
those of the pages in Mik^). With slight abuse of notation, we write the transformed state by x{k). 
Also, we use the shorthand notation Ap for ^pi,...,p„, pi € {0, 1}, i € V. Then, the distributed link 
matrices Ap in (j22p and the average link matrix A in (j23p are partitioned accordingly as 



Ar^ 



Apfic Ap^cjv 

Ap,AfC Ap,AfM_ 



Amc Ann 



(35) 



Since the time average yc has converged sufficiently by time /cQ) the proposed approach employs 
the value yc{ko) as xc{k) for all k > kQ. Hence, the value at time ko is reset as 



x{ko) 



yc{ko) 

XAf{ko) 



The updates are carried out through the distributed algorithm given by 



x{k + l) = A 



ri{k) 



x[k) -\ s, 

n 



k> h 



0, 



where 



ir,(fc) 



I 







A 



A, 



■n(k),MC ^r,{k),NN. 



:i 



I 

rn)A^(k),MC (1 - rn) Ari{k),MM, 



,s 



(36) 



(37) 



with 1 € ]R"~"'i^ °^ We note that the matrices Ap are nonnegative, but are no longer stochastic; 
the sums of the entries of the first ni(A;o) columns are larger than 1 while those of the other columns 
are smaller than 1. Hence, though x{k) > still holds, the state x{k) may not be a probability 
vector. In addition, this scheme is in the distributed form of ()2ip . and not the one in pip based 
on the modified link matrices. 
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The time average y{k) is also modified by fixing tlie entries for i G C(fco) as 



yik) 



yc{ko) 
yj\f{k) 



k > ko, 



where yj\f{k) is determined through the original formula Q. 

For the approximate update scheme ()36p . its average state x{k) := E[x{k)] follows the recursion 



m 



x(k + l) = Ax(k)-\ s, k>ko, 

n 



(38) 



where the average link matrix A := E[Aj^ri^\] is given by 
A 



I 



(39) 



/_ 0_ 

(1 - m)A^c (1 - 'm)Aj^j^_ 

Regarding this average link matrix, the following result will be useful in the subsequent devel- 
opment. 



Lemma 6.1 The submatrix Aj\j-f/ of the average link matrix A as given in ()39p satisfies the fol- 
lowing: 

(i) p{Aj\fj\/) G [0, 1 — m] and, in particular, Aj\/j\j- is a stable matrix, 
(ii) (/ - A^^)-^ > 0. 

Proof: (i) Since the original average link matrix A in ()35p is a stochastic matrix, the block 
diagonal matrix diag(0, Aj^j^) containing the submatrix A/^/^ satisfies A > diag(0, Aj^/j^/) > 0. By 
the property of nonnegative matrices |15j . it follows that 1 = p{A) > p{Aj^j^) > 0. Therefore, 
p{Aj\fj\f) = p((l — m)Aj\fj\f) G [0, 1 — m]. By Lemma [01 (i). rh G (0, 1) and hence p{Aji/ji/) < 1. 

(ii) Let A := 1 — p{A_^f_^f). This is the eigenvalue of / — A_^ff^f with the smallest real part. By 
(i), we have A > 0. Thus, / — Aj\/_\f is an M-matrixfj, so it has an inverse that is nonnegative [IB] . 
D 

We remark that in (i) in the lemma, the level of stability is affected by the parameter a as it 
determines the size of rh. 

It is clear that the value vector x* (with the coordinate change) is an equilibrium of the system 
([38|) . We partition it as 

(40) 



It is also simple to show that the vector x(A;o) given by 



xiko) 



xciko) 
xuiko) 



yc{ko) 



(41) 



{I-A^^) '{AMcyc{ko) + ^s)_ 

is an equilibrium of the system ()38p . This vector always exists by (ii) in the lemma above. 

After the pages in C{kQ) have terminated their updates, the dynamics of the scheme can be 
characterized as follows. 



'^A matrix X £ 
positive real parts. 



is said to be an M-matrix if its off-diagonal entries are nonpositive and all eigenvalues have 
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Lemma 6.2 For the distributed update scheme ([36]) and its average system ([38]) . the fohowing 
statements hold. 

(i) The average state x{k) converges to x{ko) and, in particular, xj\f{k) — )■ x_\f{ko) as k ^- oo. 
(ii) If \xc{ko) — x^\ < 5x*Q, then |5;a/'(^o) — x*j^\ < 5x^. 

Proof: (i) Since x(A;o) is an equilibrium of the average system (|38p . it follows that 



/ 



{x{k) - x{ko)). 



x{k + 1) — x{k()) 

Here, we have xc{k) — 5;c(^o) = 0, /c > /cq. Thus, 

xj^{k + 1) - x^f{ko) = Af^f^ {xM{k) - x^f{ko)) . 

By Lemma [6T] (i) . Aj^fj^ is a stable matrix. Hence, we have xj\f{k) — xj\f{ko) ^^ as A; — )■ oo. 
(ii) For the average system ([55|) . x{ko) and x* are both equilibria, and thus 

XMih) -x*j^= {I - Amm)~ Af^dxcih) - x*c). 

By Lemma [6.11 (ii). (/ — ^a^a^)~^ > 0. Moreover, by construction, Aj^j-q > 0. Thus, using the 
assumption, we have 

\xAf{ko) - xlf\ < {I - Aj^j^y A^fc \xciko) - x^l 

< {l-A^^)~^A^cSx*c 

< 6{I - A^^) iA^c x*c + —l\= 5xlf, 

where the last equality follows because x* is an equilibrium of ()38p . D 

The lemma shows that if the values in yciko) = xc{ko) are actually close to the true values 
rr^, then via the recursion in (j36|) . we can still obtain an approximate value xj\f{ko) in the average 
sense for all other states; the approximation level is the same as that for xc{ko), represented by the 
parameter 6. 

The following is the main convergence result for the scheme described in this section. 

Theorem 6.3 Consider the distributed scheme in ()36p . where under the given sample paths 
{Vi{f^)}k=o , i = I, ■ ■ ■ ,n, at time /cq, the updates at ni{ko) pages have terminated. The time aver- 
^■gs VAfik), k > ko, converges to the equilibrium xj^{k()) in the mean-square sense as £'[||y;\^(fc) — 
5?A^(fco)|| ] — )• 0, A; — > oo. 

The proof is presented in Appendix IA.3[ It is based on that for Theorem 13. 4i However, unlike 
the setup there, the distributed link matrices in the current scheme are not stochastic. This means 
that we cannot employ general Markov process results of, e.g., [11]. In contrast, the proof relies on 
the stability of the submatrix Aj^j^ as shown in Lemma l6. II (i). Also, for this reason, the analysis 
does not involve the modified link matrices such as M^^tj^\^^^^^ti^\ that appeared in the previous 
section. 
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6.2 Distributed algorithm with update termination 

We present the distributed algorithm with update termination based on the results from this section. 

Algorithm 6.4 For i £ V, page i executes the following. 

0) Initialize the parameters n, a, Xi(0), A^'^, and 5. Set k = 0, C(0) = 0, and ni(0) = 0. 

1) At time k, generate rji^k) € {0, 1} under the probability a. If rii{k) = 1, then send the value 
Xi{k) to pages j ^ C{k) that it is linked to, and request pages j ^ C{k) having links to it for 
their values. 

2) Update the value Xi{k) and its time average by 



,{k + l) = J2{^v{k))^J^Jik) + -, 



Viik) = J^^^^ii^)^ 



where A^(fc) is constructed by ([37|) using C{k). 

3) Check if yi{k) has sufficiently converged based on (|M|) . If so, then (i) add i to the set C{k), 
(ii) send yi{k) to the pages having direct links to page i, and (iii) fix Xi{i) = yi{i) = yi{k) for 

e>k. 

4) If C{k) = V, then terminate the algorithm. Otherwise, set C{k + 1) = C{k) and k = k + 1, 
and then go to Step 1. V 

We remark that, in this scheme, the choice of the parameters 6 and Ng affects the accuracy in 
the values when the pages terminate their updates as well as the time when the pages decide to 
do so. Taking 5 smaller and/or Ng larger will improve the value estimates, but will require longer 
time before the updates terminate; this in turn will keep the computation and communication load 
higher. 

7 Discussion on asynchronous iteration methods 

In this section, we discuss the application of a numerical analysis method known as asynchronous 
iteration [S] to the distributed computation of PageRank values. Deterministic algorithms for 
the PageRank problem under this approach have been discussed in, e.g., [121124). We present a 
randomization-based algorithm and clarify its relation to the schemes of this paper. 

Consider the original update scheme in (j2ip based on the power method. Let i]i{k) G {0, 1} for 
i € V, fc G Z+, be the i.i.d. random processes whose distributions are as in ()20p . Similarly to the 
scheme with simultaneous updates in Section [5l when rji{k) = 1 at time k, then page i initiates 
an update; such an event occurs with probability a. However, the difference is that this update is 

22 



performed as in the power method, and moreover pages whose corresponding processes 'r]i(k) are 
zero do not make any updates. 

The distributed update recursion is given as follows: 

x{k + 1) = M^^(fc),,„,^„(fe)x(A;), (43) 

where the initial state x(0) is a probability vector and the link matrices are given by 

:i-m)a,j + f ifK = l, 
(Mpi,...,P„),^. := <j 1 if P^ = and i = j, (44) 

otherwise. 

It is clear that these matrices keep the rows of the original link matrix M in ([3]) for the pages 
that initiate updates. Other pages just keep their previous values. Thus, these matrices are not 
stochastic. 

The following result shows that through this algorithm, we can compute the PageRank values. 

Lemma 7.1 Under the distributed update scheme of (|43p . for every initial state x(0) that is a 
probability vector, the PageRank value x* is obtained as x{k) — >■ x*, k — )• oo, with probability one. 

The distributed update scheme in (jl3]) is a randomized version of the one in [5l Section 6.2], and 
the proof can be extended in a straightforward way. Specifically, it relies on the property p{M) = 1 
that the link matrix M has. The algorithm is based on general asynchronous iteration algorithms 
for distributed computation of fixed points in the field of numerical analysis. It is interesting to 
note that the proof of the result above employs an argument similar to that of Lyapunov functions. 
We also point out that the convergence rate is exponential and the general scheme can handle 
delays in the communication. 

In comparison, the algorithms proposed in this paper have the following characteristic features. 
First, the link matrices in the update schemes ([8]) and ()2ip are stochastic, and this property is 
exploited in the convergence analysis. It moreover provides the relation to consensus type problems 
as discussed in Section [H Second, there is a difference with regard to the type of links over which 
communication takes place. In particular, it is shown in a subsequent paper [20] that the present 
approach can be extended in such a way that each page communicates only with those connected 
by outgoing links; the information of such links are by default available locally. By contrast, in the 
asynchronous iteration algorithm, pages must utilize the incoming links. This means that popular 
pages linked from many pages need extra storage to keep the data of such links. 

8 Numerical example 

We present an example with 1,000 web pages (n = 1,000). The links among the pages were 
randomly generated. The first ten pages are designed to have high rankings and are given links 
from over 90% of the pages. Others have between 2 and 333 links per page. 
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Figure 2: Sample paths of yi (solid lines) with the times at which updates stopped (marked by Q 
for each page) and the true PageRank values x* (dashed lines) for i = 21, . . . , 30. 
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Figure 3: Estimation error: ||e(A;)||i (solid line) and ||e 



(dash-dot line) 



We ran Algorithm 16.41 in Section [6] where each page initiates an update at a fixed probability 
a = 0.01 and terminates the updates when an approximate value is obtained. In the distributed 
scheme ([12|) . we generated sample paths of the processes rji{k), i € V, which determine the pages 
initiating updates, and computed the state x{k). The initial state x(0) was taken as a random 
probability vector. The parameters for the update termination were chosen as follows: The number 
Ns of steps before stopping the update was Ns = 800 and the parameter 5 determining the level of 
approximation was 5 = 0.01. We chose these values so that the characteristics of this scheme are 
visible in the plots. 

The responses of the time average yi for i = 21, . . . , 30 are shown in Fig. [5J The time when the 
corresponding pages terminated their updates are marked by Q- We observe that the convergence 
is fairly fast, and the updates stop by time fc = 4, 500 for these pages. 

Let the errors in the estimates be e{k) := y{k) — x*. In Fig. [3l these are shown for two cases: 
||e(fc)||i under the ii norm and ||e(/i;)||oo under the loo norm. The plot shows that the error in the 
individual values of yi (measured by the ioo norm) rapidly decreases and remains small while the 
total error (in the ii norm) also decreases but at a slower rate. 

In Fig. U the final values of yi for the first twenty pages are plotted as Q together with the 
acceptable ranges of error, that is, [(1 — 6)x*, (1 + S)xl] by two lines connected in the middle. As 
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Figure 4: Ranges of approximate PageRank values (marked by two lines) and yi for i = 1, . . . , 20, 
at A: = 8, 000 (marked as Q) 

we mentioned in Section [6l the time average y is no longer normalized in this case. However, the 
sum of all yi at k = 8, 000 turned out to be about 0.989, which is very close to the desired value 1. 

9 Conclusion 



In this paper, we first gave an overview of the PageRank computation problem, which is critical 
in making accurate search results with Google. We introduced a randomization-based distributed 
approach for the computation of PageRank values and showed the mean-square convergence of 
the proposed schemes. It was demonstrated that the approach has a clear relation to consensus 
type problems. The algorithms were generalized in the recent papers [191I20|. where random link 
failures and computations based on aggregating groups of pages are addressed and more discussions 
on the advantages of this approach can be found. Future research will deal with the effects of 
communication delays and the improvement of convergence rate, and also study issues related to 
implementation of the proposed distributed algorithms. 
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A Appendix 

In this appendix, following some preliminary material, the proofs of Theorems l3.4l and l6.3l are given. 
A.l Preliminaries 



We present some results related to infinite products of stochastic matrices from |13p32j . These are 
required for the proof of Theorem 13.41 given in the next subsection. 
First, the notion of weak ergodicity is introduced. 

Definition A.l Given a sequence of stochastic matrices {-P(^)}fc^o "^ M"^", let their (backward) 
product be T{k) := P{k) ■ ■ ■ P{0). The sequence {P{k)}^^Q is said to be weakly ergodic if 

tri{k) — trj{k) ^ Q, A; — > oo, Vi, j, r S {1, . . . ,n}. (45) 

In (j45p . tri{k) for all i tend to be equal as A; — t- oo, that is, all columns of the product matrix 
T{k) coincide in the limit. However, in general, the columns do not converge to a single vector. 
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To characterize matrix sequences that are weakly ergodic, we employ the tool known as the 
coefficient of ergodicity. Let r(-) be the scalar function for stochastic matrices in R"^" given by 



t{P) := - max V] \pri - Prj \ ■ (46) 

r=l 

This function t(-) has the following properties. 



Lemma A. 2 (i) t{P) € [0, 1] and, moreover, t{P) = if and only if there exists a probability 
vector t! G M" such that P = vl'^ , where 1 = [1 • • • 1]^ € M". 

(ii) t(P) = max{||Px||i : x € R", ||2;||i = 1, Yli ^i = 0}. 

(iii) t{PQ) < t{P)t{Q) for stochastic matrices P,Q. 

Weak ergodicity can be characterized by the following lemma. 

Lemma A. 3 For a sequence of stochastic matrices {P{k)}^Q, their product P{k) ■ ■ ■ P{0) is 
weakly ergodic if T{P{k)) < tq for all k, where tq G (0, 1) is a scalar. 

A. 2 The distributed update scheme 

We now analyze the proposed algorithm in Section [3 



Lemma A. 4 For the distributed update scheme ()14p and its average system ()16p . the following 
hold. 

(i) The matrices Mj, i G {1, . . . , n}, and M satisfy T{Mi) < 1 — m and t{M) < 1 — m. 

(ii) For any mode sequence {6{k)}^ the matrix sequence {Mgi^i^^} is weakly ergodic. 

Proof : (i) We show only for M since the case of Mi is similar. Recall that by (J13p and (jlSp . 
the average matrix M can be expressed as M = (1 — m)A + ^S. Thus, we have 

I " 

t{M) = -max V \{M)ri - {M)rj\ 

Z i,j '■ — ' 
r=l 



1 " — — — 

(l-m)-maxV|(^)ri-(^)rj| = (1 - m)r(^). 



2 1,3 _, 

r=l 

The matrix A is stochastic, and hence, by Lemma lA. 21 (i). it holds that t{A) < 1. Consequently, 
we arrive at the inequality t(M) < 1 — m. 

(ii) This follows from (i) and Lemma lA. 31 since m G (0, 1) by Lemma 13.31 (i). D 

Proof of Theorem \cl.4\ ' Let the error from the average be e{k) := x{k) — x*. Note that e{k) 
satisfies Y17=i^ii^) ~ ^- This is because in the systems (^T^ and (fT6|) . by assumption, the initial 
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states are probability vectors, and furthermore, Mj, i = 1, . . . ,n, and M are stochastic matrices; 
hence, both x{k) and x* are nonnegative vectors whose entries add up to 1. 
Observe that 

k , k 



f=0 l=G 



Thus, 

E[\\y{k)-x*f]=E 



1 ^' 



£=0 



1 



:E 



k-1 k- 



(fc + l)2 ^ 
^ ' '-£=0 

k-1 k-i 



Y^e{lfe{l) + 2Y,Y.<^)''<^ + 



=0 r=l 



—2 E ^[e w^<^)] + 2 E E ^[<^)^<^ + '^)] • 

'^ '-^=0 £=0 r=l J 



(47) 



We use the norm relation ||z|| < ||2;||i for z € M" [TS] and the property ||a;(-^)||i = ||a^*||i = 1 to 
obtain the bound ||e(£)|| < 2. Then, in the first summation term in ()47p . we have 



In the second summation term, we see that the summands can be written as 

E[eiife{i + r)] = E[e{if{x{i + r) - x*)] 

= E[eiif{Me^,^r-i) ■ ■ ■ M^w^W - ^*)] ■ 



(48) 



(49) 



Here, by taking the expectation of the matrix product Mg(£_|_r„x) " " " ^e{e) with respect to the 
random variables 0(i + r — 1), . . . 



E[ei£feii + r)] = E[e{if {E[Meii+r-i) ■ ■ ■ Me(,)]x(£) - x*)] 

= E[e{if{E[Mg(^,+,_,)] ■ ■ ■ £;[M,(,)]x(£) - x*)] 
= E[e{£f(Afx{i)-x*)], 



(50) 



where the second and third equalities follow from the independence oi 9{i + r — 1) , . . . ,0{i) and the 
definition of the average matrix M, respectively. Since, by Lemma [3.3l (iii). x* is the eigenvector of 
M for the eigenvalue 1, it follows that 'nT x{t) — x* = 'W {x[tj — x*). Further, we have x(£) —x* = 
e{€j and again apply the fact ||z|| < ||2;||i, z G M", to derive from ([50]) that 

E[e{lf e{l + r)] = E[e{ti^W €{(.)] < ^[||e(^)||||M''e 
<2i^n|ire(^)||il. 



(51) 

where in the last inequality, we also used ||e(^)|| < 2. As we have mentioned above, it holds that 
^11=1 Ci(^) = 0. Thus, apply Lemma [A. 21 (ii) and (iii) to ([5T]l and obtain 



E[e{lfe{i + T)]<2T{Nf)E[\\emr] 



<2T{MyE[\\emi\ <^r{M)\ 



(52) 
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where the last inequahty is due to ||e(£)||i < 2. Note that by Lemma lA. 41 t{M) < 1. 
Finally, by substituting ([MD and dS^D into (glD, we have 






<^fi + ^=V 



k + l\ 1-t(M)J 
and hence using the bound on t{M) in Lemma lA.41 (i), we obtain 

E[\\y{k)-x*f]<^^^^^^Q, k^^. (53) 

m[k + 1) 

Thus, the PageRank value x* is obtained through the time average y. D 

A. 3 Proof of Theorem [gTBl 

For simplicity, let the initial time of the update scheme (|36p to be fco + 1 = 0. Further, we write 
xj\f for xj^{ko). Denote the error between the state and the average by e(/c) := xj\f{k) — xj\f. Then, 

yuik) -XM = ■^-— ^{x}4{€) - xj^) = — y J] e{l). 

1=0 i=Q 

Thus, 

^ f k k — 1 k — / >. 

E[\\yx{k) - xmW'] = n—J^l E Ei<^f<^)] + 2 E E EWfeii + r)] . (54) 

*- "*" ) ^£=0 e=0 r=l ' 

In what follows, we must evaluate the two summation terms on the right-hand side. 
First, we claim that e{k) is uniformly bounded and in particular, for each k >0, 

||e(A;)||i < ||x^(0)||i + ^ + P^||i =: 7, (55) 

where e := maXpg{o,i}" ll^p,A^c5?c + m/n l||i. Notice that 

||e(fe)||i = \\x^{k) - xj^Wi < \\xj^{k)\\i + \\xj^\\i. (56) 

From the distributed update law in ([55]) . it easily follows that 

xj^{k + 1) = Aj^(^k),AfAfXAfik) + A^^k),MC^C H 1- 

By definition, ^,;(fc),ArA^ = (1 — rh)A^(^j^^^^j^ and ^,;(fc),ArA^ ^^ ^ submatrix of the stochastic matrix 
A^(fc). Consequently, we have ||^,;{fc),AAA/'lli < 1 — "i. Using this bound, we obtain 



\xj^{k + l)||i < ||A^(fc),ArA^||^ • ||a:AA(A;)||i + 
<{l-m)\\x^{k)\\i+e. 



Ar,{k),MCXC + — 1 
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Thus, 



fc-i 



|xAr(A;)||i < (l-m)^-||xAr(0)||i+eJ^(l 



m] 



i=o 



<\\xAfm\i + -. 

m 
Therefore, substituting this into (|56|) . we have shown 

Now, with the bound ()55|) . the first summation term in ()54p can be upper bounded as 



Y,EWYem<{k + l)^' 



(57) 



£=0 



We next look at the the second summation term of (j54p . The summands can be written as 

E[e{ti^e{i + r)] = E[e{lf[^ I]{x{l + r) - x)] 



E 



l+r-l 



e(£f[0/]L4^(f+^„i)---^^(^)x(£)+ ^ A^ 






Here, by taking the expectation of the matrix products Aj^t(^^j._i\ ■ ■ ■ A^(^£_^j\, j = 0, 1, ... ,r — 1, 
with respect to the random variables ri{i + r — 1), . . . , rj(i + j), we have 



E[e{lfe{i + r)] =E 



E 



e{£f[OI]{E[A^(^,+,_^yA^^,)]x{l) 
e+r-i 

j=£ 

e{ef[0 I] fi?[I^(,+,_i)] • • • E[A^^,)]x{e) 



m , 



-s — X 



n 



e+r-i 



+ J2 E[A^^e+r-i)]---E[A 



m , 



vU+i)\ 



-s — X 



j=l 



n 



E 



i+r-l 



e(^f [0/](l^x(£)+ Y. l^+'-i-i^s 



where the second and third equalities, respectively, follow from the independence of ?](£ + r — 
1), . . . , r]{i+j) and the definition of A in (j39]) . As x is an equilibrium of the average system in (p8|) . it 
can be shown that E[e{if e{i+r)] = E[e{if[0 I]A'^{x{£)-x)] . Here, we have x{£)-x = [0^ e{eff 
and use the norm relation ||z|| < ||z||i for all z € M" [15] to derive 



E[e{efe{i + r)] =E 



e{£f[0 I]A' 





e(£). 



= E[e{£fA^^^eii)] < E[\\em ■ ||lWe(^)||] 

< ii;[||e(^)||i . ||lWe(^)||i] < E[\\eml ■ PWIIJ 

<7^||^A^A^|ll, 



(58) 
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where in the last inequahty, we used ||e(£)||i < 7 from ()55p . The average matrix Aj^j\j- can be 
bounded as ||y4_A/'_A/||^ < 1 — m because Aj^j^ = (1 — m)Mj\fj\f, and Mj^j^ is a submatrix of a 
stochastic matrix. Thus, we arrive at 

E[e{lfe{i + r)] < 7^ (1 - mf . (59) 

Finahy, by substituting (f57|) and (f59]) into ([5^ and by m G (0, 1) from Lemma [531 (i), 

E[\\yM{k)-XMf] 

k-l k- 
1 I o 

< 






{k + 



7^ / 1 — m 

< ^^1 + 2 



fc + 1 \ m 

The far right-hand side converges to zero as A; — ?> 00, which completes the proof. D 
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